Comparison of recent Snia datasets 
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We rank the six latest Type la supernova (SnIa) datasets (Constitution (C), Union (U), ESSENCE 
(Davis) (E), GoldOe (G), SNLS lyr (S) and SDSS-II (D)) in the context of the Chevalier-Polarski- 
Linder (CPL) parametrization w{a) = wq + Wiil — a), according to their Figure of Merit (FoM), 
their consistency with the cosmological constant (ACDM), their consistency with standard rulers 
(Cosmic Microwave Background (CMB) and Baryon Acoustic Oscillations (BAO)) and their mutual 
consistency. We find a significant improvement of the FoM (defined as the inverse area of the 95.4% 
parameter contour) with the number of SnIa of these datasets ((C) highest FoM, (U), (G), (D), (E), 
(S) lowest FoM). Standard rulers (CMB+BAO) have a better FoM by about a factor of 3, compared 
to the highest FoM SnIa dataset (C). We also find that the ranking sequence based on consistency 
with ACDM is identical with the corresponding ranking based on consistency with standard rulers 
((S) most consistent, (D), (C), (E), (U), (G) least consistent). The ranking sequence of the datasets 
however changes when we consider the consistency with an expansion history corresponding to 
evolving dark energy (wo,ifi) = ( — 1-4, 2) crossing the phantom divide line w = —1 (it is practically 
reversed to (G), (U), (E), (S), (D), (C)). The SALT2 and MLCS2k2 fitters are also compared and 
some peculiar features of the SDSS-II dataset when standardized with the MLCS2k2 fitter are 
pointed out. Finally, we construct a statistic to estimate the internal consistency of a collection of 
SnIa datasets. We find that even though there is good consistency among most samples taken from 
the above datasets, this consistency decreases significantly when the Gold06 (G) dataset is included 
in the sample. 

PACS numbers: 98.80.Es,98.65.Dx,98.62.Sb 



1. INTRODUCTION 

The accelerating expansion of the universe has been 
indicated consistently by a wide range of cosmological 
data. The most sensitive probes of this expansion cur- 
rently are standard candles in the form of Type la su- 
pernovae (SnIa) i, i, i, i, d, 0, i, B [13 and stan- 
dard rulers in the form of the sound horizon scale at last 
scattering as measured through the Cosmic Microwave 
Background (CMB) power spectrum [Til. T3l a nd through 
Baryon Acoustic Oscillations fBAO) |13.|l4j. 15]. These 
cosmological observations have indicated^J, H^, Il6|, [ITf 
that the simplest cosmological model consistent with the 
observed accelerating expansion is the ACDM model [l^ 
(based on a cosmological constant) even though models 
based on dynamical dark energy |19| or modified gravity 
[20| remain viable alternatives. 

Several new SnIa datasets have emerged during the last 
3-4 years aiming at mapping in detail the accelerating ex- 
pansion in the redshift range z G [0,2]. The main such 
datasets, analyzed in the present study, are shown in de- 
tail in Table [T] of the next section and include the compi- 
lations: Constitution ^, Union i, ESSENCE (tLsNLS 
1st year (SNLSl) (1], Gold06 Q and SDSS-II [3. As 
shown in Table [T] most of these compilations include sub- 
sets obtained with different instruments. Even though 
efforts have been made in most cases to reanalyze the 
SnIa light curves and reject outliers in order to smooth 
out potential systematics due to inhomogeneities of the 
samples, systematics remain as a source of uncertainties 



[2^12 

The above SnIa standard candle data (luminous 
sources of known intrinsic luminosity) are geometric 
probes used to measure the luminosity distance ^^(z) 
which, assuming flatness, is connected to the Hubble ex- 
pansion rate H{z) as 



dL{z) 



c(l + .)£..'^, (1.1) 



where c is the velocity of light. Alternative geometric 
probes are standard rulers (objects of known comoving 
size). These may be used to measure the angular diame- 
ter distance dA{z) which, in a flat universe, is related to 
H{z) as 



d-Aiz) 



l + z 



dz' - 

Hiz'Y 



(1.2) 



The most useful standard ruler in cosmology is the 
last scattering sound horizon {z ~ 1090), the scale of 
which can be measured either directly through the CMB 
temperature power spectrum [ill [T^ . [2^ or indirectly 
through Baryon Acoustic Oscillations (B AO) on the mat- 
ter power spectrum at low redshifts [3, [ij, [l^ . These 
data lead to constraints on H{z) that are independent 
from those of standard candles. Thus the mutual con- 
sistency of the standard candle and standard ruler con- 
straints can be used as a quality test for both classes of 
data. This is one of the consistency tests implemented 
on SnIa datasets in the present study. 
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All of the above mentioned geometric probes aim at 
mapping the expansion rate H(z) as a function of the 
redshift z. The determination of the Hubble parameter 
H(z) is equivalent to identifying the function w{z) defined 
as 

wiz) = -I + (1.3) 

d\n{H{zf/H^ ~ »o.n(l + zf - r>o,(l + 
dz 

where the term in the logarithm accounts for all terms in 
the Friedmann equation not related to matter (present 
normalized density Slom) and radiation (present normal- 
ized density rJor)- If the origin of the accelerating expan- 
sion is dark energy then w(z) may be identified with the 
dark energy equation of state parameter w{z) = —. The 

cosmological constant {w{z) = —1) is the simplest dark 
energy model and corresponds to a constant dark energy 
density (ACDM model). The vast majority of presently 
available cosmological data are consistent with ACDM 
[13] at the 95.4% level (see however [IHj and references 
therein for some puzzling exceptions). 

In view of the increasing number of Snia dataset com- 
pilations a need has emerged for ranking these compila- 
tions with respect to their Figure of Merit [2^,[23| (FoM), 
defined as the inverse area of the 95% confidence region 
^, their degree of consistency with ACDM, with stan- 
dard candles and with each other. These consistency 
rankings require the derivation of suitable statistics de- 
signed to achieve them in an efficient manner. The goal 
of the present study is to provide such statistics and ap- 
ply them in order to rank the SnIa compilations of Table 
[T] of section 2 according to 

• their FoM in the context of the Chevalier-Polarski- 
Lindcr (CPL) [2^, [2^ parametrization of dynam- 
ical dark energy 

w — wq + wi{l — a) — wq + wi (1-4) 

1 + z 

• their degree of consistency with ACDM 

• their degree of consistency with standard ruler con- 
straints of Ref. [H, [3| thus testing the quality of 
the SnIa data (assuming that the distance duality 
relation ^^(z) — d^(z)(l + z)^ is applicable) 

• their degree of consistency with each other. 

In the context of ranking the SnIa compilations with 
respect to their consistency with ACDM we make use of 
the CPL parametrization and assuming flatness, we iden- 
tify the "distance" in units of ct (cr-distance da{^om)) of 
the "reference" parameter space point {wq^wi) = (—1,0) 



Note that we use the 2a contour in parameter space instead of 
the 95% confidence region used by the Dark Energy Task Force. 



corresponding to ACDM from the best fit point (wq, wi) 
of each SnIa dataset and for several priors of f2om- Simi- 
larly, in order to rank the SnIa compilations with respect 
to their consistency with CMB-BAO standard rulers we 
follow the above method but we replace the ACDM "ref- 
erence" point by the best fit (wq, wi)^^ parameter values 
obtained in the context of standard ruler (CMB-t-BAO) 
data for each rJom prior. 

In addition to the u-distance we also use the Binned 
Normalized Differences (BND) statistic of Ref. [s^l to 
rank the datasets according to their consistency with 
ACDM. Finally, in the context of measuring the inter- 
nal consistency of a set of n compilations we consider the 
mean cr-distance 

1 " 

da{^Om]Wo,Wl) = - 'S^d„i{VLQya\Wo,Wi) , (1.5) 

n ^-^ 

1=1 

where dai{^ora\WQ,wi) is the cr-distance from the ref- 
erence parameter point {wq,wi) to the best fit point 
of the compilation (see Fig. [71). Minimization of 
da{^i:im',wo,wi) at fixed fiom leads to a minimum mean 
cr-distance and to the parameters {wo^wi) of max- 
imum consistency for the particular set of compila- 
tions. Smaller ^^(f^om; u'Oi 'U'l) implies a better inter- 
nal consistency for the set of compilations. Minimizing 
da{^Om', wq,wi) with respect to fiom we can also find the 
value of f2om that maximizes the consistency among the 
SnIa data compilations. 

The above statistics are applied on the SnIa data of 
Table [T] in the following sections. In section 2 we sum- 
marize the likelihood calculations needed to evaluate the 
cr-distances described above and briefly discuss the BND 
statistic described in detail in Ref. |30i]- We also dis- 
cuss the main features of the considered SnIa datasets 
and rank them according to their FoM. In section 3 we 
apply the cr-distance statistic and the BND statistic to 
rank the data of Table [1] according to their consistency 
with ACDM and with the CMB-BAO best fit. In section 
4 we apply the mean cr-distance minimization to find the 
internal consistency of various sets of compilations ob- 
tained from Table [1] Finally in section 5 we conclude, 
summarize and discuss future prospects of the present 
study. 

2. LIKELIHOOD ANALYSIS 

Our likelihood analysis method is described in detail 
in Refs fSfl and [33|. Here we only review some of the 
basic steps for completeness. We assume a CPL [28j , 
[m parametrization for w(z) (equation (11.4^ 1 and apply 
the maximum likelihood method separately for standard 
rulers (CMB-I-BAO) and standard candles (SnIa) assum- 
ing flatness. The corresponding late time form of H{z) 
for the CPL parametrization is 

H^{z) = H^[nomil + zf + 

+ (1 - 17o„0(l + z)3(i+"'o+-i)ewf ] . (2.1) 
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In the context of constraints from standard rulers 
we use the datapoints {R, la, lOOflbh^) of Ref. [T^ 
(WMAP5) where i?, la are two shift parameters [s^ ]. 

For a fiat prior, the 5-year WMAP data (WMAP5) 
measured best fit values are 113 



V 



CMB — 



R 

L 

lOOOh/i^ 



1.710 ±0.019 
302.10 ±0.86 
2.2765 ± 0.0596 



(2.2) 



The corresponding covariance matrix is [12 



0.000367364 0.00181498 -0.000201759 \ 
CcMB = I 0.00181498 0.731444 -0.0315874 

-0.000201759 -0.0315874 0.00355323 J 

(2 



We thus define 



X 



CMB 



i?- 1.710 
la - 302.10 
lOOrJfe/i^ _ 2.2765 



(2.4) 



and construct the contribution of CMB to the as 

XCMB = XcMB CcMB """XcMB (2.5) 

with the inverse covariance matrix 

/ 2809.73 -0.133381 158.356 \ 
Ccmb"^ = -0.133381 2.21908 19.7195 (2.6) 
\ 158.356 19.7195 465.728 / 

Notice that Xcmb depends on the four parameters {ilm, 
rib, wq and wi). In what follows we use the priors h = 
0.705, nb = 2.2765/100/i2 

In the case of BAO we also apply the maximum like- 
lihood method using the datapoints of Ref. [l3| 
(SDSS5). For comparison, we have also considered the 
more recent data of Ref. [la| (SDSS7) and have found mi- 
nor differences in the results (slightly reduced consistency 
with ACDM in the context of the CPL parametriza- 
tion but no change in the ranking sequences). In some 
cases we show the results of both sets of datapoints 
(Figs. 3, 4a, 9a). 

Following Ref. |32i| we find the contribution of BAO 
to as 



2 

Xbao 



Xbao ''" Cbao ^Xbao 



(2.7) 



The analysis of Snia standard candles is also based on 
the method described in Ref. 32]. Two of the earli- 
est and reliable datasets are the Gold06 dataset [9| and 
the Supernova Legacy Survey (SNLS) [§| dataset. The 
Gold06 dataset compiled by Riess et. al. is a set of super- 
nova data from various sources analyzed in a consistent 
and robust manner with reduced calibration errors aris- 
ing from systematics. It contains 152 points from previ- 
ously published data plus 30 points with z > 1 discovered 



by the HST[9I|. Even though the Gold06 data are of high 
quality, they are plagued by non-uniformity (they are 
a collection of data obtained from various instruments) 
and they include a few outliers. Filtered versions of the 
Gold06 dataset attempting to deal with these problems 
have been included in subsequent compilations. 

The SNLS is a 5-year survey of SnIa with z < 1. The 
SNLS has adopted a more efficient SnIa search strat- 
egy involving a "rolling search" mode where a given 
field is observed every third or fourth night using a sin- 
gle imaging instrument, thus reducing photometric sys- 
tematic uncertainties. The published first year SNLS 
dataset (SNLSl) constitutes of 44 previously published 
nearby SnIa with 0.015 < z < 0.125 plus 73 distant SnIa 
(0.15 < z < 1) discovered by SNLS, two of which are 
outliers and are not used in the analysis. At this point it 
.3)should be mentioned that both SNLS and Gold06 are reli- 
able datasets, however, the non-uniformity of the Gold06 
makes it less reliable compared to SNLS which is signifi- 
cantly more uniform. 

We will also use the ESSENGE SnIa dataset of Davis 
et. al. 01 which constitutes of four subsets: ESSENGE 
[H, m (60 points), SNLSl [1] (57 points), nearby 3 
(45 points) and HST [3] (30 points) and the Union SnIa 
dataset of Kowalski et. al. Q which constitutes of 414 
SnIa, reduced to 307 points after various selection cuts 
were applied in order to create a homogeneous and high- 
signal-to-noise dataset. Finally, we will also use the Con- 
stitution SnIa dataset of Hicken et. al. [p| which consti- 
tutes in total of 397 SnIa out of which 100 come from 
the new low-z CfA3 sample and the rest from the Union 
dataset. The inclusion of the new low-z sample in 
the Constitution dataset is a major improvement over 
previous datasets because the previous sample of nearby 
SnIa was relatively small and based on early investiga- 
tions, leading to significant systematic uncertainties. In 
our analysis we use the Constitution datasets analyzed 
with both SALT and MLCS17 fitters @. 

Finally, we also consider the recently released first year 
data of the SDSS-II fl3| aiming to alleviate the lack of 
data at intermediate redshift. In our analysis we con- 
sider the largest combined sample of SnIa considered in 
[13]. This encompasses 103 SnIa from the SDSS-II, 33 
nearby SnIa [13, 56 points from ESSENCE [H, 62 from 
SNLS [| and 34 from HST |] making up a total of 288 
SnIa. Here, we consider the SDSS-II dataset analyzed 
with both the SALT2 hght-curve fitter and the MLCS2k2 
fitter '21'] and we compare the corresponding consistencies 
with ACDM . We have made simple tests to ensure that 
our analysis agrees with the one from [l^], but in the 
case of the SDSS-II dataset analyzed with the SALT2 
fitter, there is some ambiguity as the authors of Ref. [13] 
have not released the necessary covariance matrix that 
accounts for the correlations between the shape luminos- 
ity parameter (xi), the color parameter (c) and the over- 
all flux normalization (xq). 

In Table[T]we give some details about the SnIa datasets 
used in this analysis, such as the redshift range or the 
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Table 1: The datasets used in the present analysis. See respective references for details on the sources of the Snia data points. 



Dataset 


Date Released Redshift Range 


95f or anla 


Filtered subsets included 


rleis 


SNLSl 


2005 


0.015 <z< 1.01 


115 


SNLS LR 




GoldOe 


2006 


0.024 <z< 1.76 


182 


SNLSl [8|, HST [9], SCP [2], HZSST [3| 


[9] 


ESSENCE 


2007 


0.016 <z< 1.76 


192 


SNLSl [8], HST [9J, ESSENCE [35], [7] 


[35], [7] 


Union 


2008 


0.015 <z< 1.55 


307 


Gold06 [9], ESSENCE[35],[r] 


[6] 


Constitution 


2009 


0.015 <z< 1.55 


397 


Union [6], CfA3[5] 




SDSS 


2009 


0.022 <z< 1.55 


288 


Nearby [341, SDSS-II [10], ESSENCE [35|, 
SNLS f8], HST [9] 


fio] 




(b) 



* 



0.17 cr 



-1.2 
Wo 



-1.0 



Fig. 1: The 68.3%(la) — 95.4% {2a) confidence contours in the wq — wi plane based on parametrization (|2.ip for the 
ESSENCE (left) and SNLSl datasets (right) for fiom = 0.24. The arrows indicate the cr-distance of ACDM (green points: 
(wo, wi) = ( — 1,0)) to the best fit points (red points). 



subsets of each set. 

The SnIa observations use proper light curve fitters [2]| 
to provide the apparent magnitude m{z) of the super- 
novae at peak brightness. The resulting apparent mag- 
nitude m{z) is related to the dimensionlcss luminosity 
distance Dl{z) through 



mth{z) = M{M,Ho) + Uogw{DL{z)) , 



(2., 



where we have used the notation of Ref. ^] . The theo- 
retical model parameters are determined by minimizing 
the quantity 



N 



xlnIai^m,Wo,Wi) = ^ 



(2.9) 



i=l 



where N is the number of SnIa of the dataset and cr^ ^ are 
the errors due to flux uncertainties, intrinsic dispersion 
of SnIa absolute magnitude and peculiar velocity disper- 
sion. These errors are assumed to be Gaussian and un- 
correlated. The theoretical distance modulus is defined 



as 



where 



/io = 42.38 — 5logioh , 



(2.11) 



The steps we followed for the usual minimization of (|2.9p 
in terms of its parameters are described in detail in Refs. 

In order to study the consistency of the various SnIa 
datasets with the cosmological constant and the stan- 
dard rulers we consider the distance in units of a (cr- 
distance do-) of the best fit point to a model with param- 
eters {wq ,wi), where this reference point can be either 
ACDM or some other reference point, (see Fig. [IJ. The 
CT-distance can be found by converting Ax^ = X^uiq ~ 
to da, i.e. solving [s^l 



1 - r(l, AxV2)/r(l) - Erf (d,/V2) 



(2.12) 



for dcr (cr-distance), where Ax^ is the difference 
between the best- fit and the reference point {wo,wi) 
(eg ACDM) and Erf() is the error function. The 
right hand side of Eq. (|2.12p comes from integrating 



fith{zi) = mth{zi) ~ M = 5logio{DL{z)) + fiQ , (2.10) /_ 



e 2^dx, where n is the desired number of 
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Table 2: The Figure of Merit (FoM) for the datasets of Table 
[1] for fiom = 0.28 using the CPL parametrization. For com- 
parison we also show the corresponding FoM obtained with 
standard ruler data. 



Dataset 


# of Snia 


Figure of Merit 


SNLSl 


115 


0.208 


GoldOe 


182 


0.367 


ESSENCE 


192 


0.245 


SDSS-II (SALT2) 


288 


0.366 


SDSS-II (MLCS2k2) 


288 


0.553 


Union 


307 


0.512 


Constitution (SALT2) 


397 


0.708 


CMB+SDSS5 




2.028 


CMB+SDSS7 




2.541 



as, while the left hand side corresponds to the Cumula- 
tive Distribution Function (CDF) of a distribution[38] 
with two degrees of freedom. Note that Eq. (|2.12p is 
only valid for the two parameters {wo,wi) and should 
be generalized accordingly for more parameters [s^. In 
the special case of n = 1 or n = 2 we obtain the well 
known results Axio- = 2.30 and Ay 2a = 6.18 valid for 
two parameter parametrizations [38[ . 

Even though the cr-distance is not a commonly used 
statistic it is quite useful because it can directly give in- 
formation about probability of a given region in param- 
eter space. The integer values of sigma distance {la and 
2a) are commonly used to draw the corresponding con- 
tours in parameter space. We have extended this statistic 
to non-integer values in order to find the specific contours 
that go through particular reference points of parameter 
space and thus estimate quantitatively the consistency 
of these points. The advantage of using the a-distance 
instead of Ax^ is the fact that the a-distance takes into 
account the number of parameters of the parameteriza- 
tions and can therefore be directly translated into prob- 
ability for each point in parameter space. This is not 
possible for Ax^ because it does not include information 
about the number of parameters of the parameterizations 
considered. 

The Figure of Merit (FoM) is a useful measure of the 
effectiveness of a set of data in constraining cosmological 
parameters. In the case of two parameters (as for the 
CPL parametrization) it is defined as the reciprocal area 
of the 95.4% contour, in parameter space {wo,wi) 0, 
\2T\ . Clearly, the larger the FoM the more effective the 
dataset in constraining the parameters {wq, wi). In Table 
[5] we show the FoM for each dataset of Table 1 for a prior 
of 17om = 0.28. In Fig. [5] we show the FoM in terms of the 



1.0 




'100 150 200 250 300 350 400 
Number of SnIa data 

Fig. 2: The Figure of Merit (FoM) in terms of the number of 
the SnIa data for ilom = 0.28 using the CPL parametrization. 



number of the SnIa data for the same datasets. Clearly, 
the FoM is an increasing function of the number of SnIa in 
the datasets. An exception to this rule is the ESSENCE 
dataset which has a slightly smaller FoM compared to 
the Gold06 dataset even though it has a larger number 
of SnIa. 

We should stress that the FoM does not depend only 
on the total number of SnIa of the dataset but mainly 
on the number of SnIa at low and high redshifts. This 
sensitivity on the distribution in redshift space is most 
probably the origin of the ESSENCE glitch in the FoM 
plot of Fig.O Notice that the redshift space distribution 
of the ESSENCE data includes more data at intermediate 
redshifts than the Gold06 dataset while the number of 
SnIa in the ESSENCE dataset is similar to that of the 
Gold06 dataset. Finally, for comparison, in the same 
table we show the FoM corresponding to standard ruler 
data (WMAP5-hSDSS5 and WMAP5-f SDSS7). Clearly, 
the FoM of standard rulers is about a factor of 3 higher 
compared to the highest FoM of SnIa corresponding to 
the Constitution dataset. 

In addition to using to rank SnIa datasets, we con- 
sider the BND statistic 30] which is designed to pick up 
systematic brightness trends of SnIa datapoints with re- 
spect to a best fit cosmological model at high redshifts. 
It is based on binning (considering the average of) the 
normalized differences between the SnIa distance mod- 
uli and the corresponding best fit values in the context 
of a specific cosmological model (e.g. ACDM). These 
differences are normalized by the standard errors of the 
observed distance moduli (BND). As in Ref. [30] we will 
focus on the highest redshift bin and extend its size to- 
wards lower redshifts until the BND changes sign (crosses 
0) at a redshift Zc (bin size Nc). The bin size Nc of this 
crossing (the statistical variable) is then compared with 
the corresponding crossing bin size Nmc for Monte Carlo 
data realizations based on the best fit model. 

In Ref. it was found that the crossing bin size Nc 
obtained from the Union and Gold06 data with respect to 
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Dataset 
SNLSl 
SDSS-II (SALT2) 
Constitution 
ESSENCE 
Union 
SDSS-II (MLCS2K2) 
GoldOe 

CMB+BAO (SDSS5) 
CMB+BAO (SDSS7) 



„ i2o„i TOO W\ 

0.004 0.260 -1.03 0.16 

0.084 0.270 -1.09 0.51 

0.114 0.285 -0.91 -0.54 

0.227 0.270 -1.20 1.04 

0.525 0.285 -1.25 1.40 

0.623 0.360 -1.06 0.93 

0.950 0.345 -1.56 2.80 

0.200 0.272 -1.15 0.51 

0.588 0.272 -1.30 0.97 




-1.6 -1.4 -1.2 -1.0 -0.8 -0.6 



Fig. 3: Trajectories of the best fit points (uiq, w\) obtained for 
each of the datasets of Table [l] and also for the standard ruler 
CMB-BAO (WMAP5-hSDSS5 and WMAP5-^SDSS7) data as 
fiom varies in the range € [0.2, 0.36]. The arrows in the 
best fit lines indicate the direction of growing 57om. Note that 
for the SDSS5 data the standard ruler best fit parameters 
stretch out to (wq,,w\) ~ (2,-30) for $lom — 0.36, whereas 
for the SDSS7 data (too,toi) ~ (0.90, -20). 



the best fit ACDM model is anomalously large compared 
to Nmc of the corresponding Monte Carlo datasets ob- 
tained from the best fit ACDM in each case. In the next 
section we will extend this analysis to all the datasets of 
Table [Hand use the results to rank these datasets accord- 
ing to their consistency with ACDM. 



3. CONSISTENCY OF DATASETS WITH ACDM 
AND WITH STANDARD RULERS 

It is straightforward to apply the likelihood methods 
described in the previous section to find the trajectory 
of the best fit point (vjq ,w\) in parameter space as riom 
varies in the range Oom G [0.2,0.36]. These trajectories 
obtained for each of the datasets of Table [1] and also 
for the standard ruler CMB-BAO (WMAP5+SDSS5, 
WMAP5-KSDSS7) data are shown in Fig. [3 These tra- 
jectories can not be used to directly rank the datasets 
according to their consistency with any given reference 
point in parameter space (e.g. ACDM) because they con- 
tain no information about the 68.3% contours (the plot 
would become confusing if we attempted to include such 
contours). However, they provide useful hints for the 
trend of the best fit parameters as rJom varies. For ex- 
ample, such a trend is the increase of the best fit value 
of the slope w\ as the prior of fiom decreases towards 
the value 0.2 or that the best fit value of remains 
less than —1 for all datasets except of the Constitution 



Table 3: Minimum cr-distance d™*"(nS^"; -1, 0) from the 
best fit point for each of the datasets to the ACDM point. 
Also listed are the corresponding values of J7om, and the 
best fit parameters (ioo,u)i) (see also Fig. |4^). The SDSS- 
II (MLCS2K2) data showed no minimum of da with respect 
to f2om in the range f2om £ [0.2,0.36]. We thus have simply 
displayed the lowest value of do- in the corresponding range of 

f^Om. 



compilation. 

The ranking sequence of the datasets of Table [T] with 
respect to any reference point in parameter space can 
be studied quantitatively using the cr-distance statistic 
discussed above. In order to test the sensitivity of the 
ranking sequence of datasets with respect to the choice 
of consistency reference point (wq^vj\) we consider two 
such reference points: {wq,w\) — (—1,0) (ACDM) and 
(u'o,wi) = (—1.4,2) which corresponds to dynamical 
dark energy with a wi^z) that crosses the line w = —1. It 
should be noted that there is nothing special about the 
parameter point (-1.4,2). We have selected it as a repre- 
sentative of a wide region in parameter space (upper left 
from ACDM) which corresponds to dynamical dark en- 
ergy crossing the phantom divide line w = —\. Any other 
point in the same parameter region would lead to similar 
results and the same ranking of datasets. This particular 
parameter region is interesting because it is spanned by 
the best fit trajectories and it also mildly favored by the 
Gold06 dataset (see Fig. 3). 

The resulting dcr(^^om) for each dataset of Table [1] are 
shown in Figs. [3^ and[3|D in the range fiom S [0.2,0.36]. 
Clearly, there are values of Oom that minimize the o- 
distance daiS^Qm) between the best fit of each dataset and 
the reference point. These values of f^om maximize the 
consistency of the datasets with the given reference point 
in this range of riom- The minima u-distances do-(f^Om) 
for each dataset, corresponding to maximum consistency 
with ACDM along with the corresponding values of fiom 
are shown (properly ranked) in Table O The correspond- 
ing results for the reference point (wq,w\) — (—1.4, 2) are 
shown in Table [4l 

The following comments can be made with respect to 
the results shown in Figs. [3^, [lb and in the corresponding 
Tables El m 
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Fig. 4: a; cr-distances dcri{Qoin', —1,0) off the ACDM point (reference point) from tiie best fit of eacii dataset in Table[T] Notice 
that they are minimized at similar values of nom. The cr-distance do-(nom; — 1, 0) between the standard ruler best fits and 
ACDM are also shown as a function of (black and grey solid lines), b: Similar to (a) for the dynamical dark energy 
reference point {wo,wi) — (—1.4,2). Notice that the o"-distances do-i(fiom; — 1.4, 2) are minimized at more widely separated 
values of fiom. 



Dataset 




r~)min 


Wo 




GoldOe 


0.11 


0.345 


-1.56 


2.80 


Union 


0.17 


0.300 


-1.26 


1.25 


ESSENCE 


0.19 


0.275 


-1.21 


0.99 


SNLSl 


0.42 


0.290 


-1.04 


-0.26 


SDSS-II (SALT2) 


0.63 


0.275 


-1.10 


0.46 


SDSS-II (MLCS2K2) 


0.87 


0.360 


-1.06 


0.93 


Constitution 


1.77 


0.315 


-0.88 


-1.32 



Dataset 


Prob. of Consistency 


Best Fit Qom 


SNLSl 


79% 


0.26 


SDSS2-SALT2 


68% 


0.28 


SDSS2-MLCS2k2 


52% 


0.40 


ESSENCE 


30.4% 


0.27 


Constitution 


12.6 % 


0.29 


Union 


5.3 % 


0.29 


Gold06 


2.5% 


0.34 



Table 4: The minimum cr-distances d^'" to the reference 
point (uio,wi) — ( — 1.4,2), the values of at which the 
minimum distance is attained, and the best fit parameters 
{wojWi) at 0,0^" are displayed for each dataset (see also 
Fig. |4}3). We omit the rows corresponding to CMB-I-BAO 
data as the resulting a-distance is always S> 1 due to the 
dominance of the dark energy at early times. The SDSS-II 
(MLCS2K2) data showed no minimum of da with respect to 
flom in the range f2om G [0.2, 0.36]. We thus have simply dis- 
played the lowest value of do- in the corresponding range of 

Oom. 

1. The consistency with ACDM of all datasets, except 
GoldOe and SDSS-II when using the MLCS2K2 
method, is maximized in a narrow range of fiom G 
[0.26, 0.29] which also includes the value of ilom fa- 
vored by standard rulers. On the other hand, the 
consistency with the dynamical dark energy point 
{wo,wi) = (—1.4,2) is maximized over a wider 
range of fJom (f^om & [0.27,0.35]) thus decreasing 
the consistency among the datasets in the context 
of dynamical dark energy. 

2. The ranking sequence changes dramatically when 
the consistency with the dynamical dark energy is 
considered as a reference point instead of ACDM 



Table 5: Consistency of Snia datasets with ACDM according 
to the BND statistic. Notice the high f^om value favored by 
the SDSS2-MLCS2k2 dataset which is a few a above the value 
favored by other observations. 

(Table S]). Essentially the ranking is reversed! 
Thus, the choice of the consistency reference point 
plays an important role in determining the ranking 
sequence of the datasets (see also Fig. 3). 

3. The SDSS-II dataset obtained with the MLCS2k2 
fitter has some peculiar features compared to other 
datasets. In particular it favors particularly high 
values of r^om {^Om — 0.4) while for fiom < 0.3 its 
consistency with ACDM is significantly reduced to 
a level of 3cr or larger (do- > 3). In addition, the tra- 
jectory of its best fit parameter point as Qom varies 
is perpendicular to the corresponding trajectory of 
most other datasets (see Fig. 3). 

An alternative ranking of the SnIa datasets according 
to their consistency with ACDM can be made using the 
BND statistic described briefiy in section 2 and in more 
detail in Ref. [l^]- When applying the BND statistic to 
find the consistency of a given dataset with ACDM, we 
find the fraction of Monte Carlo datasets (generated from 
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Dataset 


imin 




Wo 








SNLSl 


0.003 


0.280 


-1.04 


-0.10 


-1.07 


0.08 


SDSS-II (SALT2) 


0.058 


0.280 


-1.11 


0.40 


-1.07 


0.08 


ESSENCE 


0.087 


0.275 


-1.21 


0.99 


-1.12 


0.38 


Constitution 


0.121 


0.295 


-0.90 


-0.76 


-0.84 


-1.28 


Union 


0.681 


0.280 


-1.24 


1.44 


-1.07 


0.08 


GoldOe 


1.976 


0.280 


-1.38 


2.75 


-1.07 


0.08 


SDSS-II(MLCS2K2) 


3.342 


0.285 


-0.92 


1.02 


-1.00 


-0.28 



Table 6: Consistency with standard rulers. Minimum a- 
distance d™"(f2om; (wo, i)^^) between best fit parameters 
for each dataset, {wo,wi), and best fit for standard rulers, 
{wo, wi)^'^. d^/"(f2om; {wo, wi)'^'^) is minimized at QJSl" (see 
Fig. 6). 



the best fit ACDM model) that can mimic a BND cross- 
ing redshift Zc (or crossing bin-size Nc) similar to that of 
the real data. These Monte-Carlo datasets have Zmc < 
(or equivalently A^mc ^ N^) and their fraction represents 
a probabihty of consistency of the given dataset with the 
model used to generate the Monte-Carlo datasets (best 
fit ACDM). 

An example of a distribution of crossing bin sizes Nmc 
of such Monte-Carlo datasets is shown in Fig. O for the 
case of the Constitution dataset. In this case the best fit 
ACDM model has fiom = 0.29 and it is used to generate 
222 Monte-Carlo realizations of the Constitution dataset 
as described in Ref . ^30] . The distribution of the crossing 
bin sizes Nmc of these datasets are shown in Fig. [5] along 
with the crossing bin size of the real Constitution data 
(thick dashed green line). In this case, the probability of 
consistency of the Constitution dataset with the Monte 
Carlo data of ACDM {N^c > N^) is 12.6%. 

Repeating the same process for all six datasets of Ta- 
ble [T] we assign to each of them a probability of consis- 
tency with ACDM which is shown in Table [5] Notice 
that the ranking sequence of consistency with ACDM 
obtained with the BND statistic is practically equiva- 
lent with the corresponding ranking obtained with the 
tr-distance statistic. This is reassuring for both ranking 
approaches. 

It is straightforward to apply the cr-distance statistic 
to rank the Snia datasets according to their consistency 
with standard ruler CMB-BAO data. We simply use as a 
consistency reference point the best fit point {wo,wi)^^ 
for standard rulers obtained as described in section 2 us- 
ing the WMAP5-|~SDSS5 data. In this case, the location 
of the reference point {wq, wi)^'^ in parameter space de- 
pends on r^om but this does not complicate the analysis. 
The CT-distance between the reference point {wo,wi)^^ 
and the best fit of each dataset is shown in Fig. [H] as a 
function of f7om for the datasets of Table [TJ These dis- 
tances are minimized for values of flom that are different 
for each dataset but they are all in the narrow range 
r!om e [0.27,0.3]. 



Tested Model: wo=-l,W|=0 ACDM 



Nc=58 from Constitution data (Ze-O.g). Probability N^>U^ is 12.6 




Fig. 5: The distribution of the crossing bin sizes Nmc of 222 
Monte-Carlo datasets along with the crossing bin size of the 
real Constitution data (thick dashed green line). 



SDSS2(MLCS2K2) 
__GaUDi 

Ljiion 

Constitution 
ESSENCE 
SDSS2(SALT2)._ 
SNLil--'''' 




Fig. 6: (T-distance da-i{flom; {wojWi)'^^) between the ref- 
erence point (wojWi)^^ of standard rulers (using the 
WMAP5-(-SDSS5 data) and the best fit of each dataset as 
a function of nom for the datasets of Table [T] Using the 
SDSS7 data, the minimum distances are found in the range 
S^otn G [0.29,0.31]. 



These minimum distances along with the correspond- 
ing value of riom are shown in Table [6] for each dataset 
(properly ranked according to consistency with standard 
rulers). Notice that the ranking sequence for the consis- 
tency with standard rulers is practically identical to the 
ranking sequence of the consistency with ACDM (Table 
[3]) but differs from the ranking sequence of the consis- 
tency with dynamical dark energy (Tabled. This is an 
interesting feature of the data in favor of ACDM. 

Using the SDSS7 data the distance to the standard 
rulers best fit for each dataset we find the minima shifted 
at slightly larger values of riom- We find that all of the 
minima lie in the range fiom € [0.29, 0.31] whereas the 
raking sequence of Table [6] is maintained. 

As shown in Fig. 6 the SDSS-II dataset obtained with 
the MLCS2k2 fitter has reduced consistency not only 
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Collection 


{dT") 


Var J™"' 


ES 


0.03 


0.0010 


CS 


0.13 


0.0001 


CES 


0.20 


0.0003 


CE 


0.21 


0.0022 


CUES 


0.26 


0.0043 


CUS 


0.28 


0.0093 


CUE 


0.29 


0.0147 


CU 


0.31 


0.0392 


SG 


0.45 


0.0194 


CUESG 


0.47 


0.0163 



Table 7: Table of internal consistency of dataset collections. 
The different datasets are labelled by C (Constitution), U 
(Union), E (ESSENCE), S (SNLSl)^ and G (Gold06). (C"") 
and Var d™'" stand for the average d™'" over the range fiom £ 
[0.20, 0.36] and its variance, respectively. 




-2.0 -1.5 -1.0 -0.5 



Collection 




Q 


min 
Om 


ES 





12 





265 


CS 





16 





280 


CES 





20 





275 


CE 





21 





280 


CUES 





29 





280 


CUS 





29 





280 


CU 





32 





285 


CUE 





32 





285 


CUESG 





57 





285 


SG 





93 





290 



Table 8: Minimum mean cr-distance d™'"(nj^"; -1, 0) from 
the best fit parameters of various dataset collections, to the 
ACDM parameter point. 



with ACDM (as noted above) but also with standard 
rulers. Its distance from the standard ruler best fit is at 
the level of 3 — 4cr [da — 3 — 4). This is an additional indi- 
cation of the peculiar nature of this dataset. In contrast 
the SDSS-II dataset obtained with the SALT2 fitter ap- 
pears normal and consistent with ACDM , standard ruler 
and with the rest of the datasets (see Fig. 3).^ 



^ In an effort to test the validity of our analysis for the SDSS-II 
datasets we have reproduced successfully the contours of Ref. 
[lO( (Figs. 26e and 35e) in the simplified constant w parametriza- 
tion considered there. Note that following Ref. [lO| we have only 
considered statistical errors. 



Fig. 7: Construction of da{^o-m',wo,wi) for a given point 
{■wo,w\) in parameter space. The cr-distance between the ref- 
erence point and the best fit of each Snia dataset in a partic- 
ular collection is evaluated using likelihood methods and then 
the mean a-distance is obtained using Eq. (|1.5|) . The figure il- 
lustrates the point {wo, wi) where the mean cr-distance da (ob- 
tained by collecting the Constitution and ESSENCE datasets) 
attains its minimum for a prior f2om = 0.285. The points E 
and C represent the best fit parameters for the Constitution 
and ESSENCE datasets. Also in the figure we show the 95.4% 
contours for the Constitution (dashed) and ESSENCE (dot- 
dashed) datasets, and the contour do-(f2om; two, wi) = 2 (thick, 
solid line). 



4. INTERNAL CONSISTENCY OF SNIA 
DATASET COLLECTIONS 

In this section we test the internal consistency of vari- 
ous collections of SnIa datasets using the mean cr-distance 
statistic ^^(ilom; Wo, wi) defined by Eq. (|1.5p . In Fig. [7| 
we show an example demonstrating the construction of 
da{^Om]Wi),wi) for a given point (wo,ifi) in parameter 
space. 

We minimize d„{Q.Q^]'Wo,'Wi) with respect to (wo,wi) 
and find the parameter point (wo,wi) of maximum con- 
sistency with the given dataset collection as well as the 
minimum mean cr-distance c?™'"(f2om; wo, wi) indicating 
the level of consistency. This minimized distance is 
shown in Fig. 8 as a function of r^om for various dataset 
collections. Clearly, c?™™(riom; wo, iSi) is weakly depen- 
dent on riom but depends sensitively on the dataset col- 
lection considered. 

Marginafizing c^™*"(ilom; wq, wi) with respect to fiom 
in the range ilom G [0.2, 0.36] we find (d™™) and its vari- 
ance which may be directly used as a measure of the 
internal consistency of a given dataset collection. 



10 




-CUESG 


Collection 


jmin 








- SG 


ES 


0.06 


0.280 


-1.07 


0.08 


CU 


CS 


0.16 


0.290 


-0.93 


-0.73 


CUE 












- CLIS 


CES 


0.23 


0.290 


-0.93 


-0.73 


CUES 


CE 


0.28 


0.290 


-0.93 


-0.73 




CUES 


0.36 


0.285 


-1.00 


-0.28 


CES 












- CS 


CUS 


0.41 


0.290 


-0.93 


-0.73 


ES 


CUE 


0.48 


0.285 


-1.00 


-0.28 




CU 


0.55 


0.290 


-0.93 


-0.73 




CUESG 


0.69 


0.285 


-1.00 


-0.28 




SG 


0.99 


0.280 


-1.07 


0.08 



Fig. 8: Minimum mean cr-distance d™'"(r2om; wo, wi) for the 
dataset collections considered in Table [T] The sequence of the 
legend corresponds to the sequence of lines in the plot. 



To summarize, the derivation and use of our statistic 
involves the following steps: 

1. Minimize the mean tr-distance defined by 
Eq. p.Sp . with respect to {wq,wi). 

2. Marginalize the resulting minimum mean cr- 
distance over rJom- This step is not important due 
to the weak dependence of the minimum cr-distance 
on 17om (see Fig. 8). The evaluation of the (small) 
variance with respect to Jlom is only used to demon- 
strate the weak dependence of the mean cr-distance 
on r^om and plays no further role in the analysis. 

3. Use the marginalized (over f2om) minimum (with 
respect to (wq, wi)) mean a-distance as a measure 
of the internal consistency of the dataset collection. 
The smaller the minimum mean cr-distance is, the 
'closer' together are the best fit parameter values 
in parameter space. 

The values of (d™™) and their variances for various 
dataset collections (properly ranked) are shown in Ta- 
ble [T] The following comments can be made with respect 
to the results shown in Table [T] 

• All dataset collections considered are mutually con- 
sistent in the sense that the mean a-distance be- 
tween the best fits of each dataset and the point of 
maximum consistency is less than 1 (Icr). 

• Collections including the Gold06 dataset have 
significantly less internal consistency than other 
dataset collections. 

• Maximum internal consistency is achieved for the 
ESSENCE-SNLSl collection {{d„) = 0.03) and for 
the Constitution-SNLSl collection {{d^) = 0.13) 
which are collections that also maximize consis- 
tency with ACDM and CMB-BAO standard rulers 
(see below). 



Table 9: Minimum mean cr-distance C'"(^^(C"; ("'o, ifi)^^) 
from the best fit parameters of each dataset in a particular col- 
lection to the standard ruler best fit parameters {wo,wi)^^. 



In order to rank the consistency of collections of 
datasets with ACDM and with standard rulers we con- 
sider the mean cr-distances from ACDM dai^om', —^,0) 



and from the standard ruler best fit da{^on 



„SR 



(for the standard ruler best fit {wq^,Wi^)). These are 
shown in Fig. 9 as functions of ilom- We then minimize 
these distances with respect to riom and find the cor- 
responding J™™ distances. These may now be used to 
rank the dataset collections with respect to their consis- 
tency with either ACDM or with standard rulers. These 
rankings are shown in Table [5] (with respect to ACDM 
(—1,0)) and in Table [§] (with respect to standard ruler 
best fit (w^^, wf^)). 

The following comments can be made with respect to 
the results shown in Tables [H] and [H] 

• The ranking sequences of dataset collections with 
respect to a. consistency with ACDM, b. consis- 
tency with standard rulers and c. internal consis- 
tency (Table [7]) are practically identical. This is an 
interesting result in view of the fact that the three 
ranking criteria are independent of each other. Cri- 
teria a. and c. rank the quality of the datasets 
while criterion b. ranks the consistency with a 
given cosmological model. Thus, this coincidence 
may be viewed as evidence supporting the ACDM 
model since other parameter reference points would 
in general tend to spoil this ranking (see Table [4]) . 

• More consistent collections (with any of the crite- 
ria) are those including the SNLSl dataset while 
less consistent are those including the Gold06 
dataset. 

• Maximum consistency for all dataset collections 
with either ACDM or with standard rulers is 
achieved for a narrow range of riom (^^Om G 
[0.27,0.29]). This is an indication of the robust- 
ness of the criteria used. 



11 




Fig. 9: a; cr-distances (f^Om; — 1, 0) of the ACDM point for tiie dataset collections in Table [T] The minimum values of 
dcr{rioin', —1,0) and the corresponding values of S7om are collected in Table [8] Also shown are the cr-distances between the 
standard ruler best fit (SDDS5 and SDSS7) and ACDM (black and grey solid lines), b: cr-distances da-{0,oai', {wo,Wi)^^) to the 
standard ruler best fit (WMAP5-I-SDSS5). The minimum values of dalyiom; {wojtiJi)^^) and the corresponding values of Q,om 
are listed in Table |9l 



5. CONCLUSIONS 

The main conclusions of our analysis comparing the 
six most recent Snia datasets in the context of the CPL 
parametrization may be summarized as follows: 

• All datasets can be made consistent with ACDM 
and with standard rulers at a level of 95.4% {2a) 
or less for certain prior values of the matter density 
fiom in the range fJom S [0.25,0.35]. 

• The GoldOe and the SDSS-II standardized with 
MLCS2k2 datasets have the minimum consistency 
with both ACDM and with standard rulers while 
SNLSl is the most consistent dataset with both 
ACDM and standard rulers. This may be related 
to the fact that Gold06 is highly inhomogeneous 
and includes a few outliers 22] while SNLSl is the 
most homogeneous dataset. These consistency fea- 
tures are inherited (at a reduced degree) to other 
datasets of Table 1 that include parts of SNLSl 
(e.g. Constitution or ESSENCE) or Gold06 (i.e. 
Union) . 

• The best fit of the GoldOG dataset (and of the Union 
that includes part of it) corresponds to a universe 
that super-accelerates at present {w{z = 0) — wq < 
— 1) after having crossed the phantom divide line 
(PDL) w = — 1 at a recent redshift {wi > 0) (see 
Fig. 3). On the other hand the best fit of the 
Constitution dataset gives a reversed behavior: no 
super- acceleration at present (wq > —1) but for 
f^Om > 0.25 it crosses the PDL at a recent redshift 
leading w{z) < —1 in the recent past since wi < 
(see Fig. 3). This behavior has been discussed in 
some detail in Ref . [1^ . We stress that this is only 
the behavior of the best fit and should not be taken 



as a statistically significant trend of the cosmic his- 
tory indicated by the datasets. 

• All six datasets are statistically mutually con- 
sistent. However, this consistency is somewhat 
reduced for dataset collections that include the 
GoldOe dataset (see Fig. 8 and Table [T]). This is 
consistent with previous analyses Q that pointed 
out non-uniformity systematics in the Gold06 
dataset. 

• The SDSS-II dataset obtained with the MLCS2k2 
fitter has reduced consistency with both ACDM 
and with standard rulers in contrast with the same 
dataset standardized with the SALT2 fitter which 
appears similar with the other datasets and consis- 
tent with LCDM. We have verified that this is not 
a general deficiency of the MLCS2k2 because other 
datasets using the same fitter (e.g. ESSENCE or 
the MLCS version of the Constitution dataset) ap- 
pear fairly normal and consistent (see Fig. 3). 
Therefore we have not been able to trace the ori- 
gin of the pecuHar nature of the SDSS-II MLCS2k2 
dataset. 

It is interesting that despite the improvement of stan- 
dard ruler and standard candle data quality during the 
last decade the consistency of ACDM with data has not 
decreased despite the fact that ACDM is a simple, spe- 
cific and well defined model which appears as a measure- 
zero point in all generalized models. On the contrary its 
consistency seems to be improving with time as new and 
more accurate data appear. For example, the Constitu- 
tion SnIa dataset which is a very recent compilation with 
a drastic improvement on the crucial nearby SnIa sam- 
ple, is also one of the most consistent datasets with both 
ACDM and standard rulers. 
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Despite of its excellent consistency with both Snia 
standard candles and CMB-BAO standard rulers, ACDM 
has to face potential challenges from other cosmological 
data [25II (e.g. large scale velocity flows, galaxy and clus- 
ter halo profiles, peculiar features of CMB maps etc.) 
which may lead the quest for the properties of dark en- 
ergy to interesting surprises in the near future. Such sur- 
prises may also come from future standard candle obser- 
vations or standard ruler CMB experiments (e.g. Planck 
(40| ) which are expected to significantly improve the ac- 
curacy of the constraints discussed in the present study. 

Numerical Analysis Files: The mathe- 
matica files and datasets used for the produc- 
tion of the figures may be down loaded from 
http: / /leandros. physics. uoi.gr/datacomp/ 



We have tested that these files reproduce the results of 
the original dataset papers [1, d, 0, H, 0, [l^ in the special 
case of constant w considered there. 
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